|
OpenStack Liberty : Use Cinder Storage(Multi-BackEnds)
2015/11/17 |
|
It's possible to use the Virtual Storage provided by Cinder
if an Instance needs more disks.
Configure Virtual storage with NFS and GlusterFS Multi-Backend on here.
+------------------+ +------------------+
10.0.0.50| [ Storage Node ] | 10.0.0.61| |
+------------------+ +-----+ Cinder-Volume | +-----+ GlusterFS #1 |
| [ Control Node ] | | eth0| | | eth0| |
| Keystone |10.0.0.30 | +------------------+ | +------------------+
| Glance |------------+------------------------------+
| Nova API |eth0 | +------------------+ | +------------------+
| Cinder API | | eth0| [ Compute Node ] | | eth0| |
+------------------+ +-----+ Nova Compute | +-----+ GlusterFS #2 |
10.0.0.51| | 10.0.0.62| |
+------------------+ | +------------------+
|
| +------------------+
| eth0| |
+-----+ NFS |
10.0.0.40| |
+------------------+
|
| [1] |
NFS server is required to be running on your LAN, refer to here.
On this example, configure "/storage" directory on "nfs.srv.world" as a shared directory. |
| [2] |
GlusterFS server is required to be running on your LAN, refer to here.
This example uses a replication volume "vol_replica" provided by "gfs01" and "gfs02". |
| [3] | Configure Storage Node. |
|
# install from EPEL
[root@storage ~]#
yum --enablerepo=epel -y install nfs-utils glusterfs glusterfs-fuse
[root@storage ~]#
vi /etc/cinder/cinder.conf # add follows in the [DEFAULT] section enabled_backends=nfs,glusterfs # add follows to the end
[nfs]
volume_driver = cinder.volume.drivers.nfs.NfsDriver volume_backend_name = NFS nfs_shares_config = /etc/cinder/nfs_shares nfs_mount_point_base = $state_path/mnt_nfs
[glusterfs]
volume_driver = cinder.volume.drivers.glusterfs.GlusterfsDriver volume_backend_name = GlusterFS glusterfs_shares_config = /etc/cinder/glusterfs_shares glusterfs_mount_point_base = $state_path/mnt_glusterfs
[root@storage ~]#
vi /etc/cinder/nfs_shares # create new : specify NFS shared directories nfs.srv.world:/storage
[root@storage ~]#
vi /etc/cinder/glusterfs_shares # create new : specify GlusterFS volumes gfs01.srv.world:/vol_replica chmod 640 /etc/cinder/nfs_shares [root@storage ~]# chgrp cinder /etc/cinder/nfs_shares [root@storage ~]# chmod 640 /etc/cinder/glusterfs_shares [root@storage ~]# chgrp cinder /etc/cinder/glusterfs_shares [root@storage ~]# systemctl restart openstack-cinder-volume |
| [4] | Configure Compute Node to mount NFS and GlusterFS volumes. |
|
# install from EPEL
[root@node01 ~]#
yum --enablerepo=epel -y install nfs-utils glusterfs glusterfs-fuse
[root@node01 ~]#
vi /etc/nova/nova.conf # add follows in the [DEFAULT] section osapi_volume_listen=0.0.0.0 volume_api_class=nova.volume.cinder.API systemctl restart openstack-nova-compute |
| [5] | Create volume types. It's OK to work on any node. (This example is on Control Node) |
|
# set environment variable first [root@dlp ~(keystone)]# echo "export OS_VOLUME_API_VERSION=2" >> ~/keystonerc [root@dlp ~(keystone)]# source ~/keystonerc
cinder type-create nfs +--------------------------------------+------+-------------+-----------+ | ID | Name | Description | Is_Public | +--------------------------------------+------+-------------+-----------+ | 64101cca-28d9-4a00-bebd-205eb607c697 | nfs | - | True | +--------------------------------------+------+-------------+-----------+[root@dlp ~(keystone)]# cinder type-create glusterfs +--------------------------------------+-----------+-------------+-----------+ | ID | Name | Description | Is_Public | +--------------------------------------+-----------+-------------+-----------+ | 7119e1bf-83bc-44d1-bb24-342d48db4d66 | glusterfs | - | True | +--------------------------------------+-----------+-------------+-----------+[root@dlp ~(keystone)]# cinder type-list +--------------------------------------+-----------+-------------+-----------+ | ID | Name | Description | Is_Public | +--------------------------------------+-----------+-------------+-----------+ | 64101cca-28d9-4a00-bebd-205eb607c697 | nfs | - | True | | 7119e1bf-83bc-44d1-bb24-342d48db4d66 | glusterfs | - | True | +--------------------------------------+-----------+-------------+-----------+ |
| [6] | Associates volume types set in [5] and names set in cinder.conf. |
|
[root@dlp ~(keystone)]# cinder type-key nfs set volume_backend_name=NFS [root@dlp ~(keystone)]# cinder type-key glusterfs set volume_backend_name=GlusterFS [root@dlp ~(keystone)]# cinder extra-specs-list
+--------------------------------------+-----------+----------------------------------------+
| ID | Name | extra_specs |
+--------------------------------------+-----------+----------------------------------------+
| 64101cca-28d9-4a00-bebd-205eb607c697 | nfs | {u'volume_backend_name': u'NFS'} |
| 7119e1bf-83bc-44d1-bb24-342d48db4d66 | glusterfs | {u'volume_backend_name': u'GlusterFS'} |
+--------------------------------------+-----------+----------------------------------------+
|
| [7] | Create volumes with specifying volume types. |
|
[root@dlp ~(keystone)]# cinder create --display_name disk_nfs --volume-type nfs 10
+---------------------------------------+--------------------------------------+
| Property | Value |
+---------------------------------------+--------------------------------------+
| attachments | [] |
| availability_zone | nova |
| bootable | false |
| consistencygroup_id | None |
| created_at | 2015-11-18T12:31:13.000000 |
| description | None |
| encrypted | False |
| id | e227e6ed-4741-4802-988f-c55beff7f397 |
| metadata | {} |
| migration_status | None |
| multiattach | False |
| name | disk_nfs |
| os-vol-host-attr:host | None |
| os-vol-mig-status-attr:migstat | None |
| os-vol-mig-status-attr:name_id | None |
| os-vol-tenant-attr:tenant_id | d625e02b3d394afbad250def2f88fefa |
| os-volume-replication:driver_data | None |
| os-volume-replication:extended_status | None |
| replication_status | disabled |
| size | 10 |
| snapshot_id | None |
| source_volid | None |
| status | creating |
| user_id | b4c316b93a464f8ea46b01bd01a52003 |
| volume_type | nfs |
+---------------------------------------+--------------------------------------+
[root@dlp ~(keystone)]# cinder create --display_name disk_glusterfs --volume-type glusterfs 10
+---------------------------------------+--------------------------------------+
| Property | Value |
+---------------------------------------+--------------------------------------+
| attachments | [] |
| availability_zone | nova |
| bootable | false |
| consistencygroup_id | None |
| created_at | 2015-11-18T12:31:20.000000 |
| description | None |
| encrypted | False |
| id | aef5116d-0fd2-42c2-a277-8f56b38cda02 |
| metadata | {} |
| migration_status | None |
| multiattach | False |
| name | disk_glusterfs |
| os-vol-host-attr:host | None |
| os-vol-mig-status-attr:migstat | None |
| os-vol-mig-status-attr:name_id | None |
| os-vol-tenant-attr:tenant_id | d625e02b3d394afbad250def2f88fefa |
| os-volume-replication:driver_data | None |
| os-volume-replication:extended_status | None |
| replication_status | disabled |
| size | 10 |
| snapshot_id | None |
| source_volid | None |
| status | creating |
| user_id | b4c316b93a464f8ea46b01bd01a52003 |
| volume_type | glusterfs |
+---------------------------------------+--------------------------------------+
[root@dlp ~(keystone)]# cinder list +-----------+-----------+------------------+----------------+------+-------------+----------+-------------+-------------+ | ID | Status | Migration Status | Name | Size | Volume Type | Bootable | Multiattach | Attached to | +-----------+-----------+------------------+----------------+------+-------------+----------+-------------+-------------+ | aef5116d- | available | - | disk_glusterfs | 10 | glusterfs | false | False | | | e227e6ed- | available | - | disk_nfs | 10 | nfs | false | False | | +-----------+-----------+------------------+----------------+------+-------------+----------+-------------+-------------+ |
| [8] | The Configuration is all OK. Attach volumes to instances like follows. |
|
[root@dlp ~(keystone)]# nova list +-----------+----------+---------+------------+-------------+-----------------------+ | ID | Name | Status | Task State | Power State | Networks | +-----------+----------+---------+------------+-------------+-----------------------+ | 74a217fa- | CentOS_7 | SHUTOFF | - | Shutdown | sharednet1=10.0.0.202 | +-----------+----------+---------+------------+-------------+-----------------------+[root@dlp ~(keystone)]# nova volume-attach CentOS_7 aef5116d-0fd2-42c2-a277-8f56b38cda02 auto +----------+--------------------------------------+ | Property | Value | +----------+--------------------------------------+ | device | /dev/vdb | | id | aef5116d-0fd2-42c2-a277-8f56b38cda02 | | serverId | 74a217fa-4e64-4b40-90c7-8df929a2e53c | | volumeId | aef5116d-0fd2-42c2-a277-8f56b38cda02 | +----------+--------------------------------------+[root@dlp ~(keystone)]# nova volume-attach CentOS_7 e227e6ed-4741-4802-988f-c55beff7f397 auto +----------+--------------------------------------+ | Property | Value | +----------+--------------------------------------+ | device | /dev/vdc | | id | e227e6ed-4741-4802-988f-c55beff7f397 | | serverId | 74a217fa-4e64-4b40-90c7-8df929a2e53c | | volumeId | e227e6ed-4741-4802-988f-c55beff7f397 | +----------+--------------------------------------+ # the status of attached disk turns "in-use" like follows [root@dlp ~(keystone)]# cinder list +-----------+--------+------------------+----------------+------+-------------+----------+-------------+-----------------+ | ID | Status | Migration Status | Name | Size | Volume Type | Bootable | Multiattach | Attached to | +-----------+--------+------------------+----------------+------+-------------+----------+-------------+-----------------+ | aef5116d- | in-use | - | disk_glusterfs | 10 | glusterfs | false | False | 74a217fa-4e64-4 | | e227e6ed- | in-use | - | disk_nfs | 10 | nfs | false | False | 74a217fa-4e64-4 | +-----------+--------+------------------+----------------+------+-------------+----------+-------------+-----------------+ |